Geometric Comparison of Popular Mixture Model Distances Geometric Comparison of Popular Mixture Model Distances

Triangular discrimination, Jensen-Shannon divergence, and the square of the Hellinger distance, are popular distance functions for mixture models, and are known to be similar. Here we expound upon their equivalence in terms of their functional forms after transformations, factorizations, and series expansions, and in terms of the geometry of their contours. The ratio between these distances is nearly flat for modest ratios of point coordinates, up to about 4:1. Beyond that the functions increase at different rates. We include derivations of ratio bounds, and some new difference bounds. We provide some constructions that nearly achieve the worst-cases. These help us understand when the different functions would give different orderings to the distances between points.